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Abstract 

This study investigates how cohesion manifests in readers’ 
thought processes while reading texts when they are 
instructed to engage in self-explanation, a strategy associated 
with deeper, more successful comprehension. In Study 1, 
college students (n = 21) were instructed to either paraphrase 
or self-explain science texts. Paraphrasing was characterized 
by greater cohesion in terms of lexical overlap whereas self¬ 
explanation included greater lexical diversity and more 
connectives to specify relations between ideas. In Study 2, 
adolescent students (n = 84) were provided with instruction 
and practice in self-explanation and reading strategies across 
8 sessions. Self-explanations increased in lexical diversity but 
became more causally and semantically cohesive over time. 
Together, these results suggest that cohesive features 
expressed in think alouds are indicative of the depth of 
students’ comprehension processes. 

Keywords: text comprehension, self-explanation, cohesion, 
think-aloud 

Introduction 

Comprehension is a complex cognitive activity that involves 
the processing of information for the purpose of extracting 
meaning (McNamara & Magliano, 2009). Within the 
context of text comprehension, this process relies on the 
interplay between both lower-level and higher-level 
processes. Lower-level processes, such as word decoding, 
relate to the ability to understand the surface-level attributes 
of a text. However, comprehension does not simply occur 
once these processes have taken place. Rather, deep 
comprehension relies on a reader’s ability to understand and 
make connections among the multiple concepts that are 
activated as a result of these lower-level processes. 

The outcome of these comprehension processes is 
referred to as the mental representation. This representation 
contains information that was explicitly provided in the text, 
outside information related to the text, and inferences 
generated during the comprehension process. Readers 
achieve deep comprehension when they make connections 
among these information sources and develop a coherent 
mental representation of the text (see McNamara & 
Magliano, 2009, for a review). Importantly, the coherence 
of this mental representation is established through the 
activation of prior knowledge (from within and outside the 
text), the incorporation of this knowledge into the mental 
representation, and the development of connections among 
the propositions within the mental representation. 


Many or most of these coherence-building comprehension 
processes occur online - in other words, they are enacted at 
the same time that individuals are reading the text (Kintsch, 
1988). Consequently, to better understand how readers 
establish coherence in their mental representations, these 
online processes must be identified and examined. 
Unfortunately, most text comprehension assessments only 
occur after reading is complete. While these assessments 
can measure recall and recognition of key concepts, they 
often fail to detect cognitive processes associated with 
comprehension (Magliano, 2007). 

To develop a better understanding of these online reading 
processes, researchers commonly use think-aloud 
methodologies (Pressley & Afflerbach, 1995). Generally, 
think-aloud instructions are designed to prompt individuals 
to generate the thoughts that are currently in working 
memory and thus easy to express (Ericsson & Simon, 1994). 
These methodologies have allowed researchers to identify a 
number of strategies that readers use to comprehend texts, 
such as paraphrasing and bridging (McNamara, 2004; 
Millis, Magliano, & Todaro, 2006). 

An important goal for comprehension researchers has 
been to identify factors that differentiate skilled and less 
skilled readers. Research has identified a number of 
individual differences, such as prior knowledge (McNamara 
et ah, 1996), writing ability (Allen et ah, 2014), and 
motivation (Hidi & Harackiewicz, 2000). Relevant to the 
current study, research suggests that skilled and less skilled 
readers differ in their strategic processing of texts. Skilled 
readers generate more inferences while reading, which 
allows them to connect text information to prior knowledge 
(Oakhill & Yuill, 1996). Further, skilled readers typically 
establish connections at a more global level, whereas less- 
skilled readers tend to generate connections in more local 
contexts (one or two sentences; Millis et ah, 2006). 

Researchers have additionally manipulated the 
instructional prompts in think-aloud studies to encourage 
students to engage in different cognitive processes while 
reading (e.g., Allen, McNamara, & McCrudden, 2015). For 
example, self-explanation is a commonly employed 
instructional strategy that encourages the generation of 
inferences during reading. Broadly, self-explanation is the 
process of generating explanations to oneself about a 
particular topic; this strategy has been shown to improve 
students’ deep understanding of complex concepts (Chi et 
ah, 1989; McNamara, 2004). When students produce quality 
self-explanations (either spontaneously or following an 
instructional prompt), they make inferences that link text 



content together and tie text ideas to their prior knowledge. 
Thus, self-explanation instructions can prompt students to 
behave as skilled readers. By contrast, instructions to 
generate surface level responses to a text (e.g., 
paraphrasing) do not necessarily prompt students to make 
connections that are characteristic of skilled readers. 

We consider these connections in the reader’s mental 
representation to be representative of coherence because 
research suggests that successful readers generate more 
inferences (connections) and those inferences are associated 
with enhanced comprehension. Hence, coherence is a 
construct that is indirectly inferred about the mental 
representation of the reader, generally from comprehension 
tests, but also based on computational models that simulate 
readers' performance (e.g., Kintsch, 1988). 

Cohesion, on the other hand, refers to the incidence of 
explicit text cues that facilitate readers in making 
connections among ideas in texts (Gernsbacher, 1990; 
McNamara et ah, 2014). For instance, words that overlap 
between sentences signify to the reader that the sentences 
are related. It has been assumed that cohesion cannot 
be observed in the mind of the reader, and that the elements 
of cohesion that might be related to coherence would vary 
with individual differences and the task requirements, 
among other factors (O'Reilly & McNamara, 2007). 

Our goal here is to challenge that assumption. 

Current Study 

In the current study, we examine the extent to which 
elements of cohesion detected within the (expressed) 
thoughts of readers are indicative of the type of coherence¬ 
building processes in which readers are engaging. We adopt 
a multi-step methodological approach that relies on natural 
language processing (NLP) techniques to investigate the 
cohesion of students’ think-alouds and self-explanations. In 
Study 1, we manipulate the think-aloud instructions students 
are given while reading texts, instructing them to either 
paraphrase or self-explain. We then extract the linguistic 
indices related to the cohesion of these different types of 
think-alouds that we hypothesize will be indicative of 
particular strategies. In Study 2, we examine how the 
different cohesive properties of students’ think-alouds 
change over the course of self-explanation strategy training. 
The purpose of this second analysis is to examine the extent 
to which strategy training and practice that prompts students 
to engage in deeper text processing is evident in the forms 
of cohesion established in their self-explanations. 

Cohesion in text can be established in a number of ways. 
For the sake of simplicity, we refer to three forms of 
cohesion that differ in the way in which they express 
relationships among ideas in texts. Lexical cohesion is 
cohesion that is established through overlapping words in a 
text. For instance, a text that repeats similar words will have 
higher lexical cohesion. Causal cohesion is signaled by 
overlapping actions (verbs) and connectives that serve to 
describe explicit connections among people, objects, events, 
and actions. Finally, semantic cohesion emerges from 


relationships among concepts, without relying on specific 
word overlap. For instance, two paragraphs that describe 
doctors and nurses, respectively, would still be semantically 
cohesive, even if they had no overlapping words. 

In this study, we examine the extent to which these three 
forms of cohesion manifest in students’ think alouds when 
they are engaged in tasks that lead to more coherent mental 
representations. To this end, we utilize an NLP tool, Coh- 
Metrix, to investigate the forms of cohesion present in 
students’ think-alouds. Coh-Metrix (McNamara et ah, 2014) 
calculates a number of linguistic indices, ranging from 
lower level word information to higher level information 
about cohesion. We selected Coh-Metrix indices that were 
representative of lexical cohesion (lexical overlap, lexical 
diversity), causal cohesion (connectives, causal ratio), and 
semantic cohesion (givenness, global cohesion). 

Lexical overlap measures the degree to which words and 
phrases overlap across sentences. We selected the argument 
overlap index, which specifically measures overlap between 
nouns, noun phrases, and pronouns in adjacent sentences. 

Lexical diversity is based on type-token ratios, which 
increase when the words in a text are less repetitive. In Coh- 
Metrix, lexical diversity is calculated using multiple 
algorithms that control for text length. Here, we used the D 
measure (Malvern, Richards, Chipere, & Duran, 2004). 
Lower lexical diversity is indicative of greater cohesion. In 
this context, higher lexical diversity can be associated with 
bringing in more information from outside of the text, or 
prior knowledge, when talking about the text. 

Connectives (because, therefore) specify relationships 
between ideas and provide information about the properties 
of those relationships. Coh-Metrix provides the incidence of 
connectives per 1000 words in a text. 

Causal ratio is calculated as the ratio of causal verbs to 
causal particles (McNamara et ah, 2014). The causal verb 
count is calculated with WordNet and the causal particle 
count is based on a pre-defined set of causal verbs. A higher 
causal ratio is associated with greater causal cohesion. 

Givenness measures the amount of semantic information 
that can be recovered from earlier in a text. In Coh-Metrix, 
givenness is calculated using LSA, which is a statistical 
method that uses large corpora to represent semantic 
knowledge (Landauer et ah, 2007). 

Global semantic cohesion is also analyzed using LSA. 
We utilized a paragraph-to-paragraph LSA cosine, which 
reflects the semantic similarity of paragraphs to each other. 

Study 1 

Study 1 investigates whether the form of cohesion 
established in readers’ think-aloud responses differ when 
they are prompted to self-explain or paraphrase a text. 
Paraphrases of text focus primarily on the targeted sentence, 
and thus place a less emphasis on thinking about the 
relationships to other ideas or sentences in the text. By 
contrast, self-explanations are intended to create links to 
prior knowledge and between ideas and sentences in the 
text. As such, we expected paraphrases to be characterized 



by higher lexical cohesion and a lower diversity of ideas. 
We also expected students to emphasize the causal 
connections between ideas more so when self-explaining 
than when paraphrasing. 

Method 

Participants Data from this study was collected as part of a 
larger study that examined neural correlates of strategic 
reading comprehension (Moss et al., 2011). Participants 
were undergraduate students (n = 21) recruited from large 
universities in the Northeastern United States. The majority 
of the participants were female (n = 14), with a mean 
reported age of 20.7 (SD = 2.4; range = 18-28). All 
participants were native speakers of English. 

Procedure Participants completed two sessions, which 
occurred 2-5 days apart. The first session consisted of a 
pretest and training with a self-explanation tutoring system, 
iSTART (described below). During the second session, 
participants spent 30 minutes practicing the self-explanation 
strategies in iSTART. The purpose of the self-explanation 
tutoring during these two sessions was to ensure that all of 
the participants were familiar with the strategies and had 
practiced using them while reading. Participants then read 
three separate Biology texts and, for each text, they were 
provided instructions to engage in rereading, paraphrasing, 
or self-explaining. Thus, each participant performed all 
three of the strategies. The order in which participants 
performed the strategies was randomized and the 
assignment of reading strategies to texts was 
counterbalanced across the participants. 

iSTART 

iSTART is an automated version of the SERT (McNamara, 
2004) intervention (McNamara, Levinstein, & Boonthum, 
2004). It is an intelligent tutoring system (ITS) that provides 
high school and college students with training and practice 
on reading comprehension strategies to improve self¬ 
explanation and comprehension of complex texts. 

iSTART training is separated into three modules, which 
map onto the three principles of modeling, scaffolding, and 
fading instruction. After training, students interact with the 
practice module, where direct instruction is faded and 
students are required to engage more deeply with the self¬ 
explanation strategies. Here, students are asked to self¬ 
explain target sentences in science texts and a teacher agent 
provides feedback and prompts the use of other strategies. 

Text Reading Procedure Each text was divided into three 
text sections that consisted of four paragraphs each. These 
sections were presented to students one at a time. Because 
each text was assigned a specific reading strategy, 
participants were never asked to switch strategies within a 
particular section. Each of the four-paragraph text sections 
was presented before a section of another text. For instance, 
a student might self-explain the first text section of the text 
on DNA, then reread the first text section of a text on Heat, 


and finally paraphrase the first text section on the text about 
Cells. In the next section, the student would self-explain the 
second text section on DNA, and proceed accordingly. The 
texts were divided in this manner so that no reading strategy 
was performed more than once in each of the trials in order 
to control for confounding effects, such as fatigue. 

Corpus To prepare students’ think-aloud statements for text 
analysis, we first aggregated the self-explanations and 
paraphrases into two files (this method is discussed in 
greater detail in Varner et al., 2013). Paragraph breaks were 
added to each of the aggregated files for each of the blocks 
in the trial (i.e., to preserve the three separate text sections). 
Ultimately, this yielded two aggregated files per student: 
one paraphrase file and one self-explanation file. 

Computational Analysis of Text Cohesion Students’ 
paraphrase and self-explanation files were analyzed using 
Coh-Metrix. For the purposes of the current study, we 
selected Coh-Metrix indices that were representative of text 
cohesion: lexical overlap, lexical diversity, incidence of 
connectives, causal ratio, givenness, and global cohesion. 

Results 

Our first research question regarded whether students’ self¬ 
explanations and paraphrases differed in their explicit 
markers of cohesion. A repeated-measures MANOVA was 
conducted to examine the differences in the cohesion 
indices across the aggregated paraphrase and self¬ 
explanation files (see Table 1 for descriptive statistics). This 
analysis revealed that there was a main effect of task 
instructions (i.e., paraphrase vs. self-explain) on the 
cohesion of think-alouds, F (6, 15) = 9.24, p < .001. 
Paraphrases were characterized by higher lexical cohesion, 
both in terms of higher lexical overlap, F (6, 15) = 11. 26, p 
= .003, and lower lexical diversity, F (6, 15) = 18.94, p < 
.001. Paraphrases also included somewhat greater semantic 
cohesion as measured by givenness, F (6, 15) = 3.20, p = 
.089. By contrast, more connectives were included in the 
self-explanations, F (6, 15) = 4.41, p = .049, indicative of 
greater specification of relationships between ideas. There 
were no differences between conditions in terms of the 
causal ratio or global semantic cohesion. 


Table 1: Descriptive Statistics [Means and (SD)] for 
Paraphrase and Self-Explanation Conditions 
Index _ Paraphrase _ Self-Explain 


Lexical Overlap 

0.77 (0.17) 

0.63 (0.16) 

Lexical Diversity 

50.84 (13.92) 

60.82 (9.75) 

Connectives 

88.76 (24.20) 

101.26 (15.48) 

Causal Ratio 

1.75 (1.85) 

1.19(0.85) 

Givenness 

0.49 (0.05) 

0.46 (0.04) 

Global Cohesion 

0.65 (0.09) 

0.64 (0.13) 


Discussion 

The results from Study 1 indicate that cohesion manifested 
in think-aloud statements differently as a function of 



whether students were instructed to paraphrase versus self¬ 
explain text. In particular, when readers were prompted to 
self-explain segments of the text, they used more diverse 
language and more connectives. However, given these same 
instructions, they also exhibited lower overlap among ideas, 
and less frequently referred to ideas that they had previously 
discussed. Self-explanation aims to promote the generation 
of inferences and, ideally, the establishment of connections 
between ideas in the text and prior knowledge of the world. 
When asked to self-explain, readers used more diverse 
words and less redundant information, suggesting that they 
may have been activating and thus referring to information 
from outside the text. Additionally, the self-explanation 
instructions prompted readers to utilize a greater number of 
connectives. This may suggest that when readers activated 
their prior knowledge, they used connectives to ensure that 
they explicitly established the nature of the relationships 
between the concepts they were discussing. 

Study 2 

Study 1 provides preliminarly evidence that the form of 
cohesion established in readers’ think-aloud statements can 
serve as a proxy for the processes involved in deep 
comprehension. One possibility is that these findings are 
simply a by-product of readers being explicitly told when to 
use each of the strategies. Therefore, an important question 
relates to the extent to which these different cohesion 
indices are indicative of coherence-building processes over 
the course of extended practice. 

The participants in Study 1 were relatively skilled, adult 
readers who were provided with brief self-explanation 
training with iSTART. Many readers, however, struggle to 
self-explain well (McNamara, 2004), and require extended 
self-explanation practice (Jackson & McNamara, 2013). 
Without thorough training and practice, less skilled readers 
are more likely to engage in shallow cognitive processes, 
which do not stimulate new connections among text 
concepts. Hence, our goal in Study 2 was to examine how 
cohesion manifests in the think-alouds of less-skilled (i.e., 
adolescent) readers and whether these aspects of cohesion 
change over the course of extensive training and practice. 

The data in Study 2 were collected as a part of a larger 
study that investigated the impact of self-explanation 
training on readers’ ability to generate high-quality self¬ 
explanations (Jackson et al., 2013). Results from this study 
confirmed the benefit of this training, revealing higher self¬ 
explanation scores over the course of the extended practice 
sessions. Our goal in the current study is to conduct a 
linguistic analysis of the self-explanations that students 
produced during these training sessions. The purpose of this 
analysis is to examine the cohesive devices that are related 
to students’ comprehension processes and to determine 
whether these cohesion indices can provide critical 
information about the deep comprehension processes 
developed by readers during this self-explanation training. 


Method 

Participants Participants were 84 high school students from 
a mid-south urban environment (51% male; 81% African 
American, 13% White, 6% other; average grade completed 
= 10 th grade; average age 15.8 years). All participants were 
monetarily compensated for their time. 

Procedure The current study took place over 11 sessions, 
with a pretest, 8 training sessions, a posttest, and a delayed 
retention test. Students completed a pretest during the first 
session, which included measures of their reading and self¬ 
explanation ability. During the following eight sessions, 
students received training and practice in the iSTART 
system. This study focuses on the self-explanations 
generated by the students during practice. 

Computational Analysis of Text Cohesion. Students’ self¬ 
explanations in the iSTART systems were analyzed in a 
similar manner as in Study 1 with two notable exceptions. 
First, the aggregated self-explanations preserved the 
paragraph structure of the texts in iSTART. For a target text 
with p paragraphs and n target sentences, the resulting 
aggregated self-explanation file would contain p paragraphs 
and n self-explanations corresponding to the relative 
position of the target sentence. This is because iSTART 
prompted students to self-explain at specific target 
sentences, whereas in Study 1, students self-explained or 
paraphrased entire text segments. The second difference 
relates to the calculation of the cohesion indices. Students in 
this study self-explained multiple texts per day. Therefore, 
for each student, we calculated an average score for each 
cohesion index on each day of training. 

Results 

Separate repeated-measures ANOVAs for each cohesion 
index across the 8 training days were conducted to 
investigate the effect of self-explanation training on the 
cohesion of students’ self-explanations. Table 2 presents the 
descriptive statistics for the first and last session of training. 
The results revealed that there was a significant effect of 
training session for all six cohesion indices. Over the course 
of iSTART training, students produced self-explanations 
that had lower lexical cohesion, both in terms of lower 
lexical overlap, F (1, 83) = 3.47, p = .005, and greater 
lexical diversity, F (1, 83) = 4.23, p = .001. Although the 
students decreased in their use of connectives, F (1, 83) = 
9.90, p < .001, the causal ratio increased, F (1, 83) = 3.27, p 
= .026, indicating that the students were using connectives 
that were linked to causal verbs. While the self-explanations 
included less lexical cohesion (as found in Study 1), they 
were more semantically cohesive, including more given 
information, F (1, 83) = 6.99, p < .001, and an increase in 
global semantic cohesion across training, F (1, 83) = 11.99, 
p < .001. This suggests that students improved in 
establishing connections among the self-explanations 
produced across texts (rather than simply paraphrasing 
individual sentences). 



Table 2: Descriptive Statistics [Means and (SD)] for 
Sessions 1 and 8 of Self-Explanation Practice 
Index Session 1 Session 8 


Lexical Overlap 

0.76 (0.20) 

0.61 (0.23) 

Lexical Diversity 

50.94 (13.73) 

54.29 (12.72) 

Connectives 

101.31 ( 16 . 42 ) 

85.03 (17.86) 

Causal Ratio 

0.83 (0.74) 

0.96 ( 1 . 40 ) 

Givenness 

0.35 (0.68) 

0.37 ( 0 . 09 ) 

Global Cohesion 

0.46 (0.18) 

0.62 ( 0 . 11 ) 


Note: Analyses included all 8 sessions as a repeated measure 


Discussion 

The results revealed important information about the role of 
cohesion in students’ self-explanations. The self-explanation 
training provided by iSTART promoted changes in 
students’ use of cohesion across their self-explanations. In 
particular, over time, students’ self-explanations became 
less cohesive lexically, but more cohesive at the global, 
semantic level. The explanations also included fewer 
connectives, but increased in terms of the causal ratio. The 
causal ratio is indicative of the use of connectives that are 
tied to causal verbs, and thus higher causal cohesion. These 
results indicate that self-explanation training prompted 
students to generate more inferences and establish more 
connections across the texts they were reading. This finding 
is important as it provides further confirmation that self¬ 
explanation training and practice can promote changes in 
students’ on-line comprehension processes. Further, and 
most relevant to the current study, the results indicate that 
these coherence-building comprehension processes can be 
identified (at least in part) through automated analyses of 
the forms of cohesion in students’ think-alouds. 

Importantly, however, the changes observed in the 
cohesion of students’ self-explanations across training do 
not directly map onto the findings from Study 1. In 
particular, the indices that were positively associated with 
self-explanation instructions in Study 1 did not necessarily 
increase over the course of iSTART training in Study 2. In 
Study 1, lexical overlap and givenness were higher for 
paraphrases than self-explanations, whereas connectives and 
lexical diversity were greater for self-explanations 
compared to paraphrases. In Study 2, lexical overlap and 
lexical diversity changed in a manner that was “consistent” 
with the results of Study 1, in so far as the cohesive devices 
become more indicative of deeper processing over time. 

The incidence of connectives and givenness, however, 
behaved inconsistently with Study 1. These results 
potentially point toward important differences between the 
contexts of the two studies. In Study 1, skilled readers were 
explicitly prompted to engage in different reading strategies 
before generating any text. In Study 2, however, less-skilled 
readers were practicing a host of self-explanation strategies 
over extended practice. Therefore, these students were likely 
increasing in their use of certain deep comprehension 
strategies, but in some contexts, may have also needed to 
engage in “shallow” text processing, such as paraphrasing. 


General Discussion 

The current study investigated whether the cognitive 
processes associated with deep comprehension manifested 
in the cohesive properties of students’ think-aloud 
statements. The results confirmed this prediction. In 
particular, cohesive indices of students’ think-aloud 
statements differed according to task instruction and 
changed across time as students received self-explanation 
training. These results suggest that deep comprehension 
processes can be detected through analyses of readers’ 
typed, verbal responses while reading texts. 

The results from Study 1 indicated that prompting 
students to engage in shallow text processing (i.e., 
paraphrasing) or deep processing (i.e., self-explaining) led 
them to establish different levels of cohesion in their think- 
aloud statements. In particular, when students self-explained 
texts (as compared to paraphrasing), they used more diverse 
information and established more explicit connections 
among the ideas. This finding is important for a number of 
reasons. First, it provides further confirmatory evidence for 
the fact that instructions to either self-explain or paraphrase 
a text can dramatically alter students’ on-line reading 
processes. Second, the results suggest that these 
instructional differences can be detected through analyses of 
the cohesion found in students’ think-aloud statements. 
Thus, the coherence-building processes important for text 
comprehension may manifest in the overt cohesive cues 
students use when reading through the text. 

Study 2 investigated whether the benefits of iSTART 
training could be detected through analysis of the cohesion 
of readers’ self-explanations. The results suggested that all 
of the cohesion indices changed across training days and 
that the majority became more consistent with deeper levels 
of processing (as evidenced by the results of Study 1). In 
particular, after training, students produced self¬ 
explanations that were less lexically cohesive, but more 
causally and semantically cohesive. In particular, their self¬ 
explanations contained less explicit lexical overlap, with 
greater semantic connections established across the 
statements. These results suggest that changes in coherence¬ 
building comprehension processes can be identified by 
investigating the forms of cohesion in self-explanations. 

Of course, this study is only an initial step in answering 
our questions. First, additional studies will be necessary to 
examine the relationship between these cohesive cues and 
comprehension more directly by examining students’ 
comprehension on specific texts that they have self- 
explained. In the current study, we were interested in the 
specific influence of instructional prompts on these cohesive 
cues; however, future studies investigating how these cues 
relate to comprehension on specific question types will be 
necessary. Second, further research is needed to examine the 
generality of these effects across different types of texts and 
different types of comprehension goals. 

Overall, the current study takes an important step towards 
understanding the role of cohesion in students’ think-alouds 
during text comprehension. These findings can strengthen 



our theoretical understanding of text comprehension 
processes, as well as for comprehension more broadly. 
Additionally, the results may be used to inform educational 
reading interventions and tutoring systems. If specific 
cohesion indices can be identified that systematically relate 
to certain comprehension processes and outcomes, educators 
may be able to use this information to provide more 
adaptive instruction and feedback to their students. 
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